User evaluation of Físchlár-News: An automatic broadcast news delivery system

Technological developments in content-based analysis of digital video information are seeing much progress, with ideas for fully-automatic systems being proposed and now demonstrated. Yet because we do not yet have robust, operational video retrieval systems that could be deployed and used by people the usual HCI practise of conducting a usage study and an informed iterative system design is thus not possible. Físchlár-News is one of the first line of automatic, content-based broadcast news analysis and archival systems that process broadcast news video to allow users to search, browse and play it in an easy to use manner using a conventional web browser. The system incorporates a number of state-of-the-art research components, some of which are not yet considered as mature technology, yet it has been built to be robust enough to be deployed to users who are interested in access to daily news, throughout a university campus. In this paper we report and discuss a user evaluation study conducted with 16 users, each of whom used the system freely for a 1 month period. Results from a detailed qualitative analysis are presented, looking at collected questionnaires, incident diaries and interaction log data. The findings suggest our users used the system in conjunction with their other news update methods such as watching TV news at home and browsing online news websites at their workplace, the major concerns being up-to-datedness and coverage of the news content. They tried to accommodate the system to fit their established web browsing habits, and they found local news contents and being able to play self-contained news stories on their desktop as a major value of the system. Our study also resulted in a detailed wishlist of new features which will help in further development of our and others’ systems.